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Abstract 

Formative assessment or assessment for learning is one of the most emphasized educational innovations around 
the world. Two of the common strategies that could be used in formative assessment are use of summative tests 
for formative purposes and comment only marking. We utilized these strategies in the form of formative quizzes 
in a general chemistry course and measured its effect on students’ learning. The results of our weak- 
experimental design, which was conducted with 124 pre-service elementary school teachers in an undergraduate 
level general chemistry class, showed that formative assessment alone may not be enough to improve students’ 
learning. The classroom context and background and attitude of students should be taken into account, which 
are important factors that influence student learning and achievement. In the study, two instruments and semi- 
structured interviews were used to collect data. 
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Introduction 

Assessment is an integral part of education. Assessment starts with the process of collecting information about 
learning (Harlen, 2013). This information then can be used for variety of purposes. When it is used to provide 
feedback about individuals’ learning and help them achieve learning goals, they type of assessment becomes 
formative assessment (Black and William 1998b). In other words, formative assessment refers to the assessment 
activities in which evidence about the outcomes of learning are used to improve learning. This is why formative 
assessment is also mentioned as assessment for learning in the literature (Harlen, 2013). Influential publications 
such as, OECD book. Formative assessment: improving learning in secondary classrooms, (CERI, 2005) and the 
meta analysis by Black and Wiliam (1998a), claim that formative assessment is among the most successful 
educational innovations that improve learning. 

Many strategies can be used for formative assessment in classrooms. Black and William (2009) explain that 
early work on formative assessment focused on five activities, which are 1) sharing success criteria with 
learners, 2) classroom questioning, 3) comment-only marking, 4) peer and self-assessment, 5) formative use of 
summative tests. Using quizzes for providing feedback to students is an example of formative use of summative 
tests and comment only marking strategies. There are many examples of using quizzes for formative purposes in 
many different fields, such as health sciences (Carrillo-de-la-Pena et ah, 2007), medicine (Kibble, 2007; 
Dobson, 2008), biology (Peat and Franklin, 2002), psychology (Costa, Mullan, Cothe & Butow, 2010), 
computer science (Cox and Clark, 1998), chemistry (Yalaki, 2010), and math (Lawson, 1999). Many of these 
studies utilize computers and web based quizzes for faster assessment and feedback. 

In this study, we used quizzes to offer feedback to students on their learning and help them see the areas that 
they need to improve. To use quizzes for formative purposes, we did not give grades to learners, we only 
provided written or oral feedback. Giving grades to students is not advisable in formative assessment, because it 
implies a judgment or competition and prevents students from focusing on the learning (Black and William, 
2009). The research questions we explored were: 


1. What is the effect of formative quizzes on pre-service elementary school teachers’ chemistry learning? 

2. What are the factors that influence the effectiveness of formative quizzes? 

3. What are the reactions of pre-service elementary school teachers toward utilization of the formative quizzes? 
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Method 

Participants/subjects 

In our study, we worked with 124 pre-service elementary school teachers (32 males and 92 females), which was 
a convenience sample. The study was conducted at a major university in Turkey. In Turkey, the pre-service 
elementary school teachers have to take one-semester, two credit hours chemistry class in the first year of their 
education, which is required by the national undergraduate academic program. In the general chemistry class, 
students were divided into four groups, each group containing around 30 students. This grouping was done to 
manage the class schedules and to utilize smaller classes. 


Research Methodology and Design 

The methodology of our research can be stated as action research, since the ultimate purpose of our research was 
to improve our practice. Smith (2007) explains that a common type of action research is done to improve 
personal practice, especially in educational fields. Action research does not aim for generalization since it is 
context dependent; therefore convenience sampling is a common method for defining samples. Both quantitative 
and qualitative data can be collected in action research based on research questions. 

As the authors of this study, we taught the chemistry classes to four groups of students and conducted our 
research during teaching. The research was designed as two parallel weak-experimental designs with two 
control groups and two treatment groups since there were two instructors. As the instructors, we worked 
together to teach the classes as similarly as possible. We met every week during the semester to plan for next 
classes and evaluate the past classes. Consequently, we combined the control group and treatment group data for 
analysis. 

The treatment groups received four non-graded, written quizzes during instruction, while the control groups 
were not given any quizzes. Detailed feedback was given to quizzes in in the form of written or oral comments 
in the treatment groups. Peer assessment and group discussions were also used which followed by instructor 
explanations. All groups had two midterms and one final examination during the course of the study. Students in 
the treatment groups were given a chance to review their midterm exam papers with oral feedback by reviewing 
questions and answers. Students in the control groups were not denied feedback whenever they asked for it. 


Data Instruments 

Both qualitative and quantitative data were collected in this study. To collect quantitative data, we developed 
two instruments; one was a test to measure students’ general chemistry knowledge which contained 36 
questions that required written answers. The other one was a survey instrument with a Likert type scale to 
measure students’ interests toward chemistry and the chemistry course they were taking and the formative 
assessment techniques used during the class. The instrument contained 20 items for control group students, 26 
items for treatment group students, four open ended questions for control group students, and 5 open ended 
questions for treatment group students. The difference in number of questions was due to the extra questions 
asked about the quizzes to the treatment group students. We have done validity and reliability studies of both 
instruments. The reliability coefficient for the General Chemistry Achievement Test (GECAT) was KR21= 
0.825 and the Cronbach Alpha value for the Interest Toward Chemistry (ITC) instrument was 0.895. 

In order to achieve content validity for the GECAT, we first developed a table of the course objectives and then 
decided how many questions we would ask for each objective and at what level. This way we tried to ensure 
content validity. Secondly, we have done a pilot study of the test and revised some of the questions based on the 
pilot study data. Finally we asked for three chemistry educators’ opinions about the test to ensure validity. The 
GECAT instrument was applied before and after one semester of chemistry class (14 weeks in between) as pre 
and posttest. 

To ensure validity of the ITC instrument, we asked for four colleagues’ opinions about the instrument and 
revised some of the items based on their suggestions. We field tested the instrument with a group of students 
and determined its reliability before administering it to the research participants toward the end of the semester. 
We asked students not to write their names on the questionnaire to help them feel comfortable about writing 
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their responses without fear of instructor reaction. 107 students have returned the ITC instrument with their 
responses. 

Finally, we conducted semi structured interviews with nine students. Together with the open-ended questions in 
the ITC instrument, interviews provided the qualitative data for the study. The interviewees were voluntary 
students who accepted to be interviewed for this study. The interview questions were chosen to learn about 
students’ attitudes toward chemistry and their opinions about the chemistry course they took and also the use of 
formative assessment during the course. The interviews were recorded and transcribed. 


Data Analysis 

Independent samples t-test was used to compare the pre and post test scores of the treatment and control groups. 
First, we compared the pretest scores to see if there was a difference between control and treatment groups in 
terms of their chemistry knowledge in the beginning of the semester. Then we compared the post test scores to 
see if there was a difference between control and treatment groups at the end of the class. 

Frequency tables of the items in the ITC instrument were generated to analyze the survey data. The qualitative 
data obtained through open ended questions in the ITC instrument as well as the interviews were analyzed by 
developing coding categories. Data was surveyed to develop codes for pieces of data that could be grouped 
together. As the authors of this study, we developed the codes separately and then compared. We negotiated the 
codes we developed and finally came up with 12 codes. These codes were used to organize the qualitative data 
and to make sense of it based on the framework suggested by Bogdan and Biklen (2002). 


Results and Discussion 

The independent samples t-test showed that there was no significant difference between pretest scores of control 
groups and treatment groups (p> 0.05, Table 1 and 2), which indicated that both groups were similar in their 
chemistry knowledge at the beginning of the study. The posttest scores also yielded no significant difference 
between control groups and treatment groups at the end of the study (p> 0.05, Table 3 and 4). Although there 
was a significant improvement when pre and post tests were compared, the gain was similar for both the control 
and treatment groups. 


Table 1. Group statistics for pretest GECAT instrument 


Group N 

Treatment 65 

Control 54 

Mean 

6.28 

7.33 

Std. Deviation 

4.765 

4.638 

Std. Error Mean 

.591 

.631 

Table 2. Independent samples t-test for control and treatment groups (pretest GECAT instrument) 


t-test for Equality of Means 



Sig. (2-tailed) 

Mean Difference 

Std. Error Difference 

Equal variances assumed 

.225 

-1.056 

.867 

Table 3. Group statistics for posttest GECAT instrument 

Group N 

Mean 

Std. Deviation 

Std. Error Mean 

Treatment 65 

25.37 

5.533 

.686 

Control 59 

25.10 

5.701 

.742 

Table 4. Independent samples t-test for control and treatment groups (posttest GECAT instrument) 


t-test for Equality of Means 



Sig. (2-tailed) 

Mean Difference 

Std. Error Difference 

Equal variances assumed 

.791 

.268 

1.009 


Data obtained from the ITC instrument showed that the most participating teacher candidates had negative 
feelings toward chemistry. This attitude was clear in survey data and also interview data. Table 5 provides a 
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sample of ITC survey item with student responses. One of the items in ITC instrument was the statement “I am 
interested in chemistry” to which 52.3% percent of students responded as “I disagree” or “I totally disagree,” 
20.6% of the students responded as “I am not sure,” which means totally 72.9% of the students did not approve 
this statement. To another item which stated, “Chemistry is one of the easy classes for me” 72.9% of the 
students disagreed and another 17.8% was not sure. However, this does not mean that students did not value the 
chemistry class and what they learned in the class. Even though they were not interested in the subject, they 
thought they are going to need the information they learned in the chemistry classes (at least part of it) in the 
future. So they agreed to the item “This course will be useful to me in the future in my job” with a total rate of 
67.3%. Participants provided parallel answers to similar items in Table 5. 

Students showed their negative feelings toward chemistry in their responses to the open ended questions as well. 
One of the open ended questions was “If you were to summarize this class in three words, what would you 
say?” Most of the responses to this question were negative. Students used the word “difficult” (52 out of 107) 
and “boring” (30 out of 107) most often in their responses. On the other hand, the total number of all o positive 
words that they used was just 20. The open ended questions in the ITC instrument revealed further details about 
students’ feelings, opinions, and knowledge about chemistry and also their opinions about the chemistry class 
they took. The data about students’ feelings towards chemistry was grouped under the code “feelings toward 
chemistry.” For example, in response to the survey question, “What would you like to say about this class other 
than your responses to the other questions in this questionnaire?” one student wrote “Most of us have chosen to 
study social sciences and maybe most of us have chosen this department just because of our antipathy toward 
chemistry classes.” One of the interviewed students, Ayla (pseudonym) had this to say about her friends, “Since 
my friends do not like science classes, they have chosen to study social sciences. They feel hate towards science 
fields. They come to the chemistry class with prejudices.” 


Table 5. A sample of ITC instrument items with student responses (% frequency) 


N= 107 

I Agree 
(Total) 

I am not 

sure 

I disagree 
(Total) 

This course will be helpful to me in my future career. 

67.3 

15.9 

16.8 

This course involves information that I can use in my daily life. 

64.5 

21.5 

14.0 

I value what I learn in this course. 

65.4 

21.5 

13.1 

I am interested in chemistry. 

27.1 

20.6 

52.3 

Chemistry course exceeds my capacity. 

30.8 

18.7 

50.5 

This course contributes to my career. 

67.3 

21.5 

11.2 

Chemistry is a subject that interests me. 

46.7 

17.8 

35.5 

Chemistry is an easy subject for me. 

9.4 

17.8 

72.9 


Another code we developed during data analysis was “prior chemistry knowledge” and we grouped many 
students’ responses that referred to their prior knowledge in chemistry under this code. For example, one of the 
students wrote in response to the question “What would you like to say about this class other than your 
responses to the other questions in this questionnaire?” the following answer: “Subjects are taught too fast. They 
are not explained simple enough. We have no knowledge in chemistry and this fact is forgotten sometimes.” 
Another student wrote, “I have no background knowledge in chemistry and that is why after I solve questions 
[in class] I can’t solve similar questions [later on].” Similar data was also obtained from the interviews. One of 
the interviewed students, Dora (pseudonym) said, “Our level is very low and you could not come down to that 
level. That is why we had difficulties. You should have explained things with simpler examples.” 

Still another code that we developed was “classroom teaching” under which data that related to students 
responses about our teaching methods were grouped. Majority of the students thought that more sample 
questions should have been solved during the classes. They specifically make a distinction in chemistry 
questions as “numerical” and “knowledge” questions. Numerical refers to questions that require use of math or a 
calculation and knowledge questions are the ones that require remembering information. They inherited this 
terminology from the university entrance examination jargon, which may or may not reflect the true nature of 
questions in the examination itself. Many students made the claim that mostly knowledge questions were asked 
in quizzes while they faced with numerical questions in the exams. For example, in response to the question 
“How this class could be taught in a more useful and more conducting way for your learning?” one student 
wrote: “Especially in the first quizzes, instead of only wording and memorization questions, more numerical 
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questions should have been asked. ... More numerical question could have been solved in the class.” In fact 
only two questions out of 10 required simple math in the first midterm examination. However, students were so 
afraid of such questions; many of them were just turned off when they saw them. One student wrote: 
“Numerical questions are asked in the exam but in the first two quizzes none was asked. I could not solve the 
numerical questions in the exam and that just demoralized me and affected my whole exam.” In response to the 
questions “What are your opinions about the quizzes conducted during the class? Do you think quizzes were 
useful for you learning? How the quizzes should be conducted so that they are more useful?” another student 
wrote, “Quizzes are good. However, it would have been better if they were exactly parallel to the exams. In 
exams there are both numerical and wording questions; however, quizzes are either numerical or wording.” 
Most students stated in their responses that more numerical examples should have been solved in the class. 
Since the class was limited to two hours per week, students were right that not enough time have been spared for 
solving more examples of the so called numerical questions. However, the qualitative data showed that what 
most students meant by saying “more numerical questions should have been solved in the classes” was really 
that “more questions that are very similar to the exam questions should have been solved in the classes.” 

One of the important data that was obtained from the interviews was related to the educational tradition that 
students came from that favors memorization. This data was organized under the code “educational tradition.” 
Memorization was how most students were studying and preparing for the exams. One of the interviewed 
students, Layla (pseudonym), said “For example you gave a question in the [quiz], if you make a small change 
in the question we cannot solve it, [because] we did not understand the logic of it. Because of this, more 
examples should be solved in the classes.” Another interviewed student, Bruce (pseudonym), said very openly 
“You know that we come from a tradition of memorization. We don’t like to use our brains. We want 
everything just given to us. If you make a slightest change in a question, we can’t solve it because we 
memorized it in its original form.” 

Students had very positive opinions about the quizzes we used during the semester. 68.5% of them agreed with 
the item “The quizzes conducted in this class helped me learn.” 70.3% of them agreed with the item “The 
quizzes conducted in this class were very useful.” Some of the interviewed students explained that quizzes have 
increased their frequency of studying for the chemistry course. The data provided here is a sample of what we 
obtained in our study which we think provides enough evidence to answer our research questions. 


Conclusion 

From the findings (Table 1 through 4), we can answer our first research question “What is the effect of 
formative quizzes on pre-service elementary school teachers’ chemistry learning?” The answer is “it does not 
have a positive effect in every case.” However, the reason for this is hidden in our answer to the second research 
question “What are the factors that influence the effectiveness of formative quizzes?” The answer for this 
question is much more complex and largely comes from the qualitative data. 

Survey and interview data show that students participated in this study have mostly negative feelings toward 
chemistry. This fact affected the learning process significantly in the classroom. In relation to this, one 
unexpected experience in this study for us was that in the third quiz, almost all of the students in one of the 
treatment groups said that they cannot answer any of the questions and they did not even try. When asked the 
reason, they said that they had another exam for which they had to study and could not look up any chemistry 
subjects. When asked whether they remember anything from the classes, the answer was “no.” This was a 
surprise to us because we expected students to utilize the quizzes as an opportunity for noticing their 
weaknesses and improve their learning using the provided feedback. It was a frustrating experience for us since 
no feedback can be given to unanswered questions. So, in these situations, the students were asked to work in 
groups and try to solve quiz questions together. They were asked to help each other and check each other’s 
answers. What we observed often during administering quizzes was that because of their negative attitudes 
toward chemistry, some students easily gave up on trying to learn. 

As the instructors of these classes we accept that we failed to notice how foreign chemistry was to most of the 
students in our classes and how little knowledge they brought about chemistry from high school. Our 
assumption that most students understood what we taught in the classes was wrong most of the time. In response 
to the ITC question, “How this class could be taught in a more useful and more conducting way for your 
learning?” one of the students wrote “You did not start from zero as you claimed in the beginning. You taught 
as if we all know [something]. ... Since you started like this I was lost in all subjects.” 
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Coming from an educational system that depends on high stakes testing, students were very much used to 
memorization and expected to face almost identical questions with quiz or example questions in exams. It is 
hard for them to break away from the traditions of the educational system that they come from and utilize the 
opportunities provided by formative assessment methods. Based on these findings and discussion, one of the 
conclusions of this study is that formative assessment alone may not be enough to improve students’ learning in 
challenging classroom contexts such as the ones described in this study. Smith and Gorard (2005) also found in 
their study that formative assessment was not effective in improving student progress. However, there are many 
studies in the literature claiming the otherwise. We feel that the attitudes and background of students should be 
taken into account, which are important factors that influence student learning and achievement other than they 
type of assessment used. 

Regarding our last research question, “What are the reactions of pre-service elementary school teachers toward 
utilization of the formative quizzes?” as stated in the findings, students had very positive attitudes toward the 
use of quizzes. They also thought that the number of four quizzes per semester, as we did in this study, is ideal. 
However, this positive attitude toward quizzes and the feedback given did not translate into improved test 
scores. 


Recommendations 

Strategies to make the class content more appealing and relevant to students should be utilized together with 
formative assessment in classes where student attitude toward the course is negative. Students should also be 
educated about the use of formative assessment to help them to take advantage of the opportunities brought by 
formative assessment before its utilization. 
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